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ABSTRACT 

Fair allocation has been studied intensively in both economics and 
computer science, and fair sharing of resources has aroused re- 
newed interest with the advent of virtualization and cloud comput- 
ing. Prior work has typically focused on mechanisms for fair shar- 
ing of a single resource. We provide a new definition for the simul- 
taneous fair allocation of multiple continuously-divisible resources. 
Roughly speaking, we define fairness as the situation where every 
user either gets all the resources he wishes for, or else gets at least 
his entitlement on some bottleneck resource, and therefore cannot 
complain about not getting more. This definition has the same de- 
sirable properties as the recently suggested dominant resource fair- 
ness, and also handles the case of multiple bottlenecks. We then 
prove that a fair allocation according to this definition is guaran- 
teed to exist for any combination of user requests and entitlements 
(where a user's relative use of the different resources is fixed). The 
proof, which uses tools from the theory of ordinary differential 
equations, is constructive and provides a method to compute the 
allocations numerically. 

Categories and Subject Descriptors 

D.4.1 [OPERATING SYSTEMS]: Process Management— 5c/iedi//- 
ing- K.6.2 [MANAGEMENT OF COMPUTING AND INFOR- 
MATION SYSTEMS]: Installation Management— /'ncing and re- 
source allocation 

General Terms 

Management, performance 

Keywords 

Resource allocation, fair share, bottlenecks 

1. INTRODUCTION 

Fair allocation is a problem that has been widely studied both in 
economics and computer science. In economics, a wide range of is- 
sues have been studied, ranging from the design of voting rules and 

*Much of this work was done while the author was on sabbatical 
leave at Hebrew University. 



the apportionment of representation in Congress to the allocation 
of joint costs and fair cake cutting to envy-free auctions. (See (7] 
I17II36I for a sample of the wide-ranging work in the area.) In com- 
puter science, besides the work on economics-related issues, fair 
allocation has been the focus of a great deal of attention in operat- 
ing systems, where fair-share scheduling is a major concern. (See 
the related work in Section[2]) 

But what exactly does fair allocation mean? Generally speaking, 
the notion of fairness may pertain to mechanisms like bargaining 
and their relationship to ethical issues (e.g. 1351 ). We focus on a 
more technical level, and take "fair allocation" to mean "an alloca- 
tion according to agreed entitlements". The source of the entitle- 
ments is immaterial; for example, they could result from unequal 
contributions towards the procurement of some shared computa- 
tional infrastructure, or from the dictates of different service-level 
agreements. Still, what does it mean to say that a user is "entitled 
to 20% of the system"? Is this a guarantee for 20% of the CPU cy- 
cles? Or maybe 20% of each and every resource? And what should 
we do if the user requires, say, only 3% of the CPU, but over 70% 
of the network bandwidth? Reserving 20% of the CPU for it will 
cause obvious waste, while curbing its network usage might also 
be ill-advised if no other user can take up the slack. 

Our goal in this paper is to define a notion of fair allocation when 
multiple, continuously-divisible resources need to be allocated, and 
show that a fair allocation according to our definition is guaranteed 
to exist. Our motivation comes from work in operating systems, so 
many of our examples and much of the discussion below is taken 
primarily from that literature. But, as should be clear, our approach 
is meaningful whenever a number of users need to share a number 
of different resources, each has a certain pre-negotiated entitlement 
to a share of the resources, but each has different needs for each of 
the resources. 

The common approach to resource management in both operating 
systems and virtual machine monitors (VMMs) is to focus on the 
CPU. Scheduling and allocation are done on the CPU, and this in- 
duces a use of other devices such as the network or disk. However, 
the relative use of diverse devices by different processes may be 
quite dissimilar For example, by trying to promote an I/O-bound 
process (because it deserves more of the CPU than it is using), we 
might turn the disk into a bottleneck, and inadvertently allow the 
internal scheduling of the disk controller to dictate the use of the 
whole system. Thus the CPU-centric view may be inappropriate 
when the goal is to achieve a predefined allocation of the resources. 

In order to avoid such problems, it has recently been suggested that 
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fair-share scheduling be done in two steps |14| : first, identify the 
resource that is the system bottleneck, and then enforce the desired 
relative allocations on this resource. The fair usage of the bottle- 
neck resource induces some level of usage of other resources as 
well, but this need not be controlled, because there is sufficient ca- 
pacity on those resources for all contending processes. 

The question is what to do if two or more resources become bottle- 
necks. This may easily happen when different processes predom- 
inantly use distinct resources. For example, consider a situation 
where one process makes heavy use of the CPU, a second is I/O- 
bound, while a third process uses both CPU and I/O, making both 
bottlenecks. We consider such situations and make the following 
contributions. 

We propose a definition of what it means to be fair that is appropri- 
ate even when different users or processes have different require- 
ments for various resources. The definition, presented in Section[3] 
extends the idea of focusing on the bottleneck; it essentially states 
that we are fair as long as each and every user receives his enti- 
tlement on at least one bottleneck resource. This is claimed to be 
fair because, given an assumption that each user uses the differ- 
ent resources in predefined proportions, the definition implies that 
users cannot justifiably complain about not getting more. We then 
prove in Section|5]that an allocation that satisfies our fairness defi- 
nition is guaranteed to exist. The proof is constructive and provides 
a method to compute the allocation numerically. Perhaps surpris- 
ingly, the proof makes use of tools from the theory of ordinary dif- 
ferential equations. To the best of our knowledge, this is the first 
time that such tools have been used to answer questions of this type. 

While there has been extensive work on fair allocation of resources 
over the years, there seems to be very little work that like us tackles 
the fair allocation of multiple resources of distinct types. A very 
recently suggested approach to this problem is dominant resource 
fairness, where allocations are set so as to equalize each user's max- 
imal allocation of any resource |16| . We discuss the similarities and 
differences between this scheme and ours in Section|4] 

2. PRIOR WORK 

To put our contributions in context, we first review prior work in 
fair allocation of resources in systems. The issue of resource allo- 
cation has been studied for many years, but mostly from different 
perspectives than the one we use. 

The requirement for control over the allocation of resources given 
to different users or groups of users has been addressed in several 
contexts. It is usually called /a/r-j/iare scheduling in the literature, 
where "fair" is understood as according to each user's entitlement, 
rather than as equitable. Early implementations were based on ac- 
counting, and simply gave priority to users who had not yet re- 
ceived their due share at the expense of those that had exceeded 
their share 12 II 1221 . In Unix systems, one approach that has been 
suggested 11311201 is to manipulate each process's "nice" value to 
achieve the desired effectQ Simpler and more direct approaches 
include lottery scheduling 1341 or using an economic model 1331 . 
where each process's priority (and hence relative share of the re- 
source) is expressed by its share of lottery tickets or capital. 

Another popular approach is based on virtual time 1121 1271 . The 

'"Nice" is a user-controlled input to the system's priority calcula- 
tions. It is so called because normal users may only reduce their 
priority, and be nice to others. 



idea is that time is simply counted at a different rate for different 
processes, based on their relative allocations. In particular, schedul- 
ing decisions may be based on the difference between the resources 
a process has actually received and what it would have received if 
the ideal processor sharing discipline had been used 1 5 8 , 1 5 1 . This 
difference has also been proposed as a way to measure (un)fairness 
that combines job seniority considerations with resource require- 
ments considerations |2l l30| . 

In networking research, control over relative allocations is achieved 
using leaky bucket or token bucket metering approaches. This is 
combined with fair queueing, in which requests from different users 
are placed in distinct queues, which are served according to how 
much bandwidth they should receive II0II26I . The most common 
approach to fairness is max-min fairness, where the goal is to max- 
imize the minimal allocation to any user 1291 . 

Focusing on virtual machine monitors, Xen uses a credit sched- 
uler essentially based on virtual time, where credits correspond to 
milliseconds and domains that have extra credit are preferred over 
those that have exhausted their credit 1281 . Note, however, that 
domains that have gone over their credit limit may still run, as in 
borrowed virtual time II2I . VMware ESX server uses weighted fair 
queueing or lottery scheduling I1QII34I . The Virtuoso system uses 
a scheduler called VSched that treats virtual machines as real-time 
tasks that require a certain slice of CPU time per each period of 
real time I24II25I . Controlling the slices and periods allows for ad- 
equate performance even when mixing interactive and batch jobs. 

The main drawback of the approaches mentioned above is that they 
focus on one resource — the CPU, or in a networking context, the 
bandwidth of a link. The effect of CPU scheduling on I/O is dis- 
cussed by Ongaro et al. 1281 and Govindan et al. 1181 . For ex- 
ample, they suggested that VMs that do I/O could be temporar- 
ily given a higher priority so as not to cause delays and latency 
problems. However, the interaction of such prioritization with al- 
locations was not considered. Similarly, there has been interesting 
work on scheduling bottleneck devices other than the CPU (4] II9I 
1321 , but this was done to optimize performance of the said device, 
and not to enforce a desired allocation. 

Few works have considered dealing with multiple resource con- 
straints. Diao et al. 1111 suggested an approach of controlling ap- 
plications so that they adjust their usage, rather than to enforce an 
allocation. Fairness in the allocation of multiple resources was ad- 
dressed by Sabrina et al. I3II in the context of packet schedul- 
ing, where the resources were network bandwidth and the CPU 
resources needed to process packets. The approach taken was to 
consider the processing and transmission times together when us- 
ing a weighted fair queueing framework. The interaction between 
scheduling and multiple resources was discussed by Amir et al. fTl . 
However, the context is completely different as they consider tar- 
gets for migration in the interest of load balancing. Interestingly, 
the end result is similar to our approach, as they try to avoid ma- 
chines where any one of the resources will end up being highly 
utilized and in danger of running out (and becoming a bottleneck). 
Control over multiple resources was also considered at the microar- 
chitectural level by Bitirgen et al. (5), but with a goal of achieving 
performance goals rather than predefined allocations. 

Our work extends a recent suggestion to focus on bottleneck re- 
sources |5] [T4l . Specifically, the suggestion was to identify at 
each stage which device is the system bottleneck (that is, the de- 
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vice whose usage is closest to 100% utilization) and then enforce 
the desired allocation on this device. For example, if the disk is 
the bottleneck, one can promote or delay requests from different 
users so as to achieve the desired relative allocation of bandwidth 
among them. This, in turn, induces corresponding usage patterns 
on other devices including the CPU. But if the disk is the bottle- 
neck, the other devices will be less than 100% utilized, and there- 
fore scheduling them is less important. However, this suggestion 
does not deal with what to do if there are in fact multiple bottle- 
necks, which, as we observed, can easily happen. Our work extends 
the definition to cover the case of multiple bottlenecks. 

In networking, allocations to flows traversing multiple links are 
also typically viewed as using multiple resources, where again the 
constraints stem from links that become saturated (and hence a bot- 
tleneck). In this context min-max fairness can be characterized 
based on a geometrical representation that is very similar to ours 
1291 . However, the requirements from all the resources (links) are 
equal, making the search for a solution easier. Specifically, it is of- 
ten possible to move in a straight line from the origin to the bound- 
ary, in a direction based on the desired relative allocations, rather 
than using a more complicated trajectory as we do in Section [J!4l 

To the best of our knowledge, the only other work to suggest and 
analyze a fair-share allocation policy that handles diverse require- 
ments for multiple resources is the recently proposed dominant re- 
source fairness |16| . This does not explicitly consider bottlenecks, 
but rather focuses on each user's maximal usage of any single re- 
source. We describe this in more detail and compare it with our 
definition in Section|4l 

3. SHARING MULTIPLE RESOURCES 

Fair sharing of resources has been one of the objectives of schedul- 
ing for many years, and has received renewed interest in the con- 
texts of virtualization and cloud computing. But what exactly is 
"fair sharing"? Consider a setting with A'^ users and m resources 
(CPU, network bandwidth, disk usage, and so on). We assume that 
each user i is entitled to a fixed percentage Ci of the full capacity, 
and hence of each resource, where ei + ■ ■ ■ + ejv = 1. Each user 
i requests a fraction rij of resource j. If rij < Ci for all j — that 
is, if i requests less than his entitlement on each resource — then 
any reasonable notion of fair sharing should grant user i all that he 
requests on each resource. 

But what if user i requests more than his entitlement on some re- 
source j? In this case, if rij + ■ ■ ■ + rjvj < 1 for each resource j, 
so that no resource is a bottleneck, then any efficient notion of fair 
sharing should give each user all that he requests. Even if a user 
requests more than he is entitled to of some resource, as long as no 
resource is a bottleneck, there is no problem. Clearly the problem 
arises only when rij + ■ ■ ■ + tnj > 1 for some resource j. If 
there is only one bottleneck resource, again it seems easy enough 
to cut back those users who are requesting more than their entitle- 
ment II4I . But what if there are several bottlenecks? What should 
"fair allocation" mean in this case? 

The problem is compounded by the fact that different users have 
different requirements for the different resources. For exam- 
ple, in an operating system setting, if a certain process is entitled to 
50% of the resources, but this is an I/O-bound process that hardly 
uses the CPU, the scheduler cannot force it to use more and fill its 
allocation. Moreover, reserving 50% of the CPU for this process 
will likely just waste most of this capacity. However, if we allocate 



the unused capacity to another process, which also turns out to be 
I/O-bound, we may end up hurting the performance of the original 
process. We therefore need to find a set of allocations that allow us 
to exploit complementary usage profiles to achieve high utilization, 
but at the same time respect the different entitlements. By respect- 
ing the entitlements, the allocations can be claimed to be fair. In 
particular, we define fairness by invoking the user's point of view 
of the entitlements: 

Fairness Dehnition 

An allocation of multiple resources is fair if users have no justi- 
fication to complain that they got less than they deserve. 



A key contribution of this paper is to define the properties of an al- 
location that satisfies this definition, i.e. one where any complaints 
would be unjustified. We then go on to prove that such an alloca- 
tion is in fact achievable, for any combination of requirements and 
entitlements. The discussion above already illustrates the core of 
our approach: a focus on bottleneck resources. This approach is 
in line with basic results in performance evaluation, as it is well 
known that the bottleneck device constrains system performance 
(this is, after all, the definition of a bottleneck) 1231 . An important 
manifestation of this result is that, in a queueing network, most of 
the clients will always be concentrated in the queue of the bottle- 
neck device. This implies that scheduling the bottleneck device is 
the only important activity, and moreover, that judicious scheduling 
can be used to control relative resource allocations. 

Precisely this reasoning led to the recent suggestion that propor- 
tional resource allocation be exercised on the bottleneck device at 
each instant, rather than on the same device (e.g. the CPU) at all 
times Is] 1141 . Focusing on the bottleneck in this way avoids try- 
ing to control allocations based on an irrelevant tuning knob, and 
provides the most reasonable interpretation of enforcing resource 
allocations in a multi-resource environment. 

But what happens if there are two or more bottlenecks? In order 
to derive the allocations, we first need to define a model of how 
resources are used. Given the definitions of entitlements and re- 
quirements above, our task is to figure out how much to cut each 
user back. We assume that we cut each user back by the same factor 
Xi on each resource. This is in fact our main assumption: 



Proportional Resource Usage Assumption 

Users use diverse resources in well-defined proportions. Thus 
cutting back on one resource by a certain factor will lead to 
reduced usage of other resources by the same factor 



This assumption reflects a model where each user is engaged in 
a specific type of activity with a well-defined resource usage pro- 
file. For example, a user may be serving requests from clients over 
the Internet. Each request requires a certain amount of computa- 
tion, a certain amount of network activity, and a certain amount of 
disk activity. If the rate of requests grows, all of these grow by the 
same factor. But if one resource is constrained, limiting the rate of 
serving requests, this induces a similar reduction in the usage of all 
other resources (see demonstration in Fig.[T). This is essentially the 
"knee model" of Etsion et al. il5l , where I/O activity is shown to 
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Figure 1: Illustration of a resource request profile, and how it is 
cut baci when usage of resource 4 is Umited to 50%. 



be linearly proportional to CPU allocation up to some maximal us- 
age level. It also corresponds to the task model of Ghodsi et al. 1161 
when all tasks that a user wants to execute have identical resource 
requirements (which is indeed the specific model they use in their 
proofs). Note, however, that this is indeed a limiting assumption. 
Specifically, it excludes usage patterns where one resource is used 
to compensate for lack of another resource, as happens, for exam- 
ple, in paging, or when using compression to reduce bandwidth. 

All the above leads to the following problem definition. We want 
to find x\, . . . ,xn, such that < a;^ < 1, where for each user 
i, Xi is the fraction of that user's request which will be granted. 
Furthermore, we require that 



for each resource j : xiVij 



+ XnTN] < 1. 



(1) 



This means that the total usage of each resource is limited by the 
resource capacity. Those resources for which equality holds are the 
bottleneck resources. 

Among all the allocations xi, . . . , xn that satisfy Eq. l[T), which 
should qualify as "fair"? This is where we define the "no justified 
complaints" condition: 



No Justified Complaints Condition 



A user cannot justify complaining about his allocation if either 
he gets all he asked for, or else he gets his entitlement, and giv- 
ing him more would come at the expense of other users who 
have their own entitlements. 



Using the notation above, the "no justified complaints" condition 
can be formally expressed as: 

for all users i : a;^ = 1 or 

there exists a bottleneck resource j* such that XiVij* > e^. 

(2) 

Specifically, user i cannot complain if there exists some bottleneck 
resource j* where he gets at least what he is entitled to. Since 
j* is already utilized to its full capacity, giving him more, that is, 
increasing Xi, would necessarily come at the expense of other users, 
who have the right to their own entitlements. Therefore, increasing 
j's allocation at their expense would be unfair. 

Note that it may happen that a user receives less than his entitlement 
on other resources, including other bottleneck resources, where the 
entitlement would seem to indicate that a larger allocation is man- 
dated. This is where the proportional resource usage assumption 
comes in. Recall that the factor Xi is common to all resources. 



Thus giving a user a higher allocation on any resource implies that 
his allocation must grow on all resources. The original bottleneck 
resource j* thus constrains all allocations, even on other bottleneck 
resources or resources that are not themselves contended. 

Being based on bottlenecks, it is easy to see that allocations that 
satisfy Eqs. 0} and l[2j are Pareto optimal (but, of course, not every 
Pareto-optimal solution satisfies our fairness criterion). Showing 
that such a fair allocation exists turns out to be surprisingly nontriv- 
ial. The obvious greedy approach does not seem to work. Given a 
collection of requests and entitlements, suppose that we try to sat- 
isfy the users one at a time, so that, after the fcth step, we have 
an allocation (xi, . . . ,xn) satisfying Eq. (TJ such that the first k 
users have no complaints (that is, Eq. (|2} holds for users 1, . . . , fc). 
To see why doing this does not seem helpful, suppose that there 
are three resources and three users. User 1 requests (|, 5, |) (i.e., 
rii = i, ri2 = |, and ris = |) and is entitled to 0.5 of the re- 
sources (i.e., ei = ^). User 2 requests (|, |, ^) and e2 = |, and 
User 3 requests (1, 1, i) with 63 = |. We start by giving user 1 
everything he asks for, and users 2 and 3 nothing; that is, we con- 
sider the allocation x = (1, 0, 0). Clearly at this point User 1 has 
no complaints. Next we try to satisfy User 2. If we do not cut back 
User 1, then we must have 2:2 < |, since resource 3 then becomes 
a bottleneck. But with the allocation x — (1, |, 0) User 2 has a 
justified complaint: the only resource on which he gets at least his 
entitlement is resource 2, but resource 2 is not a bottleneck with 
this allocation. So User 2 feels that he is entitled to a bigger share 
of the resources. There are various ways to solve this problem. For 
example, we could consider the allocation (|, 1, 0). It is easy to 
check that neither User 1 nor User 2 has a justified complaint with 
this allocation. But now we need to add User 3 to the mix. To do 
so, we have to cut back either User 1 or User 2, or both. It follows 
from our main theorem that this can be done in a way that none of 
the users has a justified complaint. But the naive greedy construc- 
tion does not work; at each stage, we seem to have to completely 
redo the previous assignment. Although a more clever greedy ap- 
proach might work, we suspect not; a more global approach seems 
necessary. We will describe such an approach in Section|5] 



4. PROPERTIES OF ALLOCATIONS WITH 
NO JUSTIFIED COMPLAINTS 

In their analysis of dominant resource fairness, Ghodsi et al. II6I 
show that it possesses four desirable attributes, under the assump- 
tion that all tasks that a user wants to execute have identical re- 
source requirements (in which case their model reduces to ours). 
We now show that our definition possesses them as well. This an- 
swers Ghodsi et al.'s question of whether there are other fair allo- 
cation schemes with these properties. 

The first attribute is providing an incentive for sharing: the allo- 
cation given to each user should be better than just giving him his 
entitlement of each resource (actually they defined this requirement 
only in the case that all users are viewed as having equal entitle- 
ments, in which case this amounts to giving each user ^th of each 
resource). Suppose that if user gets a fraction Ci of each resource, 
he can perform a fraction a:: of his requests, where xvij < . In an 
allocation that is far in our sense, user i gets to perform a fraction 
y of his reuqests, where yvij — d for some resource j. Thus, we 
must have x < y, which means that player i is at least as well off 
participating in the scheme as he would be if he got his entitlement 
on each resource. 
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The second attribute is being strategyproof. This means that users 
won't benefit from lying about their resource needs. Asking for less 
than the real requirements obviously just caps the user's potential 
allocation at lower levels. Asking for more with the same profile 
(that is, same relative usage of different resources) does not have an 
effect, except that the user might be allocated more than he can use. 
While this may lead to waste, it does not provide any benefit to the 
user. Modifying the profile will either give the user extra capacity 
he can't use on some resource, or worse, reduce the effective al- 
location because some unneeded resource was inflated and tricked 
the system into thinking it has satisfied the user's entitlement. Thus 
lying cannot lead to benefits, but can in fact cause harm to a user's 
allocation. 

The third attribute is that the produced allocation be envy free: no 
user should prefer another user's allocation. This follows from be- 
ing strategy proof; otherwise a user could lie about his requirements 
so as to mimic those of the other user. 

The fourth and final attribute is Pareto efficiency. This means that 
increasing the allocation to one user must come at the expense of 
another. As noted above, this follows from doing allocations based 
on bottlenecks. 

We now turn to comparing our definition of fairness with dominant 
resource fairness. While similar in spirit, the two definitions are 
actually quite different in their philosophy. At a very basic level, the 
notion of fairness depends on perception of utility. In the context 
of allocating resources on computer systems, the utility is typically 
unknown. Consequently the notion of fairness is ill-defined. 

To better understand the difference between utility and allocation, 
we recount an example used by Yaari and Bar-Hillel f35J. Jones 
and Smith are to share a certain number of grapefruit and avocados 
to obtain certain vitamins they need. They have different physio- 
logical abilities to extract these vitamins from the different fruit. 
The overwhelming majority of those polled agreed that the most 
fair division is one that gives them equal shares of extracted vita- 
mins, despite being quite far from being equal shares of actual fruit. 
But such considerations would be impossible if you do not know 
their specific ability to extract vitamins, and that they actually only 
eat fruit for their vitamins. 

When allocating resources to virtual machines or users of a cloud 
system, we do not know the real utility of these resources for the 
users. We are therefore forced to just count the amount of resources 
being allocated. The difference between definitions of fairness is in 
how this counting is done. In asset fairness, the fractions of all re- 
sources used are summed up. Thus if a user gets 20% of the CPU, 
7% of the disk bandwidth, and 37% of the network bandwidth, he 
is considered as having received 64/300 of the total resources in 
the system. In order to be fair, other users should also get simi- 
lar total fractions. In dominant resource fairness, only the largest 
fraction is considered. Thus, in the example above, the user's dom- 
inant resource is the network, and he is considered to have received 
resources at a level of 37/100. To be fair, other users should re- 
ceive similar levels of their respective dominant resources. In our 
definition of fairness, we do not focus on the dominant resource 
of each user, but rather take a system-wide view based on bottle- 
necks. Thus, if the CPU happens to be the only bottleneck, we say 
that this user received resources at a level of 20/100. The fact that 
he received more of another resource, namely the network, is im- 
material, because there is no contention for the network. A user 
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Figure 2: Example of the effect of imposing equal shares of a bot- 
tleneck resource, compared with dominant resource fairness. 



is welcome to use as much of any resource for which there is no 
contention as he likes. 

Interestingly, Ghodsi et al. prove that under dominant resource fair- 
ness each user will be constrained by some resource that is a bot- 
tleneck II16I . However, their fairness criterion does not depend 
on this bottleneck, while ours does. The following example may 
help to illustrate the differences (Fig.|2](. Consider a scenario with 
three users and two resources. The requirements of the users are 
ri = (1,0.2), r2 = (1,0.2), and ra = (0.4,0.8). The entitle- 
ments are ei = 62 = es = |. Obviously resource 1 is a bot- 
tleneck, so the allocations with our definition of fairness will be 
'^1 ~ (§' = (|, and as = (i, |). This is fair on the 

bottleneck resource, and each user receives his entitlement. Domi- 
nant resource fairness, in contrast, leads to the following allocation: 
ai = (0.4, 0.08), aa = (0.4, 0.08), and ag = (0.2, 0.4). User 3's 
usage of resource 2 is counted, despite the fact that there is no con- 
tention for resource 2; this leads to a reduced allocation of resource 
1 . There seems to be no criteria by which to say that one allocation 
is fairer than the other. It may well be that user 3 derives much 
benefit from using resource 2, and therefore cutting him back on 
resource 1 is perfectly justified. But given that we do not know that 
this is the case, we suggest that it is safer to focus on the bottleneck 
resources. 

In fact, Ghodsi et al. do mention bottleneck fairness in their de- 
scription of dominant resource fairness, but only as a secondary 
criterion. They define bottleneck fairness only when all users have 
the same dominant resource, essentially reducing the scope to the 
single bottleneck case. Our work is the first to extend this with 
a meaningful definition of fairness for multiple bottlenecks, and 
when the dominant resources are different. 

We now turn to a few more observations of the relationship be- 
tween our definition and dominant resource fairness. First, we ob- 
serve that if all users have the same dominant resource, dominant 
resource fairness and our definition are equivalent. This follows 
since the common dominant resource is the only bottleneck. 

Another interesting question is one of utilization. In the example 
given above, our definition of fairness led to higher overall utiliza- 
tion than dominant resource fairness. It this guaranteed to always 
be the case? The answer is no, as the following counter-example 
demonstrates. Assume two users and four resources, with require- 
ment vectors of ri = (i, 0, 0, 1) and r2 = (1, 1, 1, 0) and equal 
entitlements ei — 62 ~ ^. With our no justified complaints defi- 
nition, the first and last resources are the bottlenecks, and the allo- 
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cations are ai = (i, 0, 0, 1) and 02 = (|, |, ^, 0). If there were 
many more "middle" resources, the average utilization would tend 
to ^ . With the dominant resource fairness scheme, the allocations 
are ai = (i, 0, 0, |) and 02 — (|, |, |, 0). In this case, the aver- 
age utilization tends to | . 

Another important difference between the two definitions is that 
dominant resource fairness allocations can be found using an in- 
cremental algorithm 1161 . Finding allocations based on the no jus- 
tified complaints idea is harder, because we do not know in ad- 
vance which resources will be the bottlenecks. Nevertheless, the 
proof presented in the next section shows that such an allocation 
always exists. Moreover, the trajectory argument used is actually 
somewhat similar to how allocations are constructed for dominant 
resource fairness. 

5. EXISTENCE OF A FAIR ALLOCATION 

In this section, we prove that an allocation satisfying (TJ and ^ 
always exists. Note, however, that there is an additional require- 
ment that Xi < 1, and that {2) makes a distinction between the case 
Xi < 1 (need to exceed entitlement on some bottleneck resource) 
and the case Xi = 1 (get all you want). We can treat these two 
cases uniformly by defining dummy resources that are each re- 
quested by only one user. Using m' to denote the number of real 
resources, we define the requirements on the dummy resources to 
be ri^^i^i — 1 for i = 1, . . . , A^, and rij = for i = 1, . . . , A'', 
j — m' + 1, . . . , m' + N , and j 7^ m' + i. These dummy resources 
can only become a "bottleneck" if their corresponding Xi = 1, 
meaning that the user gets all he requested. In the following, m 
will denote the full set of resources including the dummy ones. 

With this addition, we want to prove the following: 

Theorem 1. Given 

• entitlements ei, . . . , ejv such that ei + ■ ■ ■ + ejv = 1 and 
Ci > Ofor i — 1, . . . ,N, and 

• resource requirements rtj, i = 1, . . . , N , j = 1, . . . , m such 
that r\j + ■ ■ ■ + rjvj > Iforj = 1, . . . ,m and < rij < 1 
for i = 1, . . . , N and j = 1 , . . . , m, 

there exists an allocation Xi, . . . , Xn, where < a;; < 1 for i = 
1 , . . . , A'^, that satisfies the two conditions 

(1 ) for all resources j, j = 1, ... ,m, XiTij + ■ ■ ■ -\- Xmtnj < 1; 

(2) for all users i, i = 1, . . . , A'^, there exists a resource j* such 
that XiVij* > Ci and xirij* + ■ ■ ■ + xnTnj* = 1. 

As the mathematical derivation is somewhat involved, we first pro- 
vide an argument for the special case N — 2 (two users); this 
enables us to draw the constructions used in 2D. The full proof for 
all values of A'^ is given in Section lS^ 

5.1 Simplifying Assumptions 

Before proving the theorem, we make three simplifying assump- 
tions, all without loss of generality. First, as reflected in the defini- 
tion of the resource requirements, we assume that, for each resource 
j, ri.j + ■ ■ ■ + TNj > 1. If there is any additional resource for 
which this inequality does not hold, we can ignore resource and 
solve the problem for the remaining resources. Whatever solution 
we come up with will also be a solution when we add back to the 
picture, because its usage will be at most r^^jo + • • ■ + rjvjo < 1. 

Second, we assume that, for each user i, there is at least one re- 
source j such that Tij > Ci. (This pertains to only real resources. 



not the dummy resources.) If this is not the case, we give user i 
everything he asked for, remove his requests, renormalize the en- 
titlements of the remaining users so that they still sum to 1, renor- 
malize the remaining capacity of the different resources so that it 
is still I, and renormalize the remaining requests by the same fac- 
tors. For example, suppose that users 1, 2, and 3 are entitled to 0.5, 
0.2, and 0.3 of capacity, respectively. If User 1 never asks for more 
than 0.5 of any resource, then we give him what he asks for, and 
remove his requests from the picture. Note that this means that, for 
each resource r, the fraction of r available is at least as much as the 
entitlement of each user. We then multiply User 2 and User 3's en- 
titlements by 2 (= 1/(1 — 0.5)), so that their entitlements still sum 
to 1. After this normalization, they are entitled to 0.4 and 0.6 of 
what remains after we have granted User I's request. Moreover, if 
User I requested, say, 0.4 of Resource 1, so that 60% of Resource I 
is still available, we multiply each of the remaining user's requests 
by I (= 1/0.6). Again, if we solve the resulting problem, we will 
have solved our original problem. This follows in general since, if 
User 1 is the one eliminated, the entitlements of the remaining users 
effectively grew by a factor of 1/(1 — ei), while the requests and 
capacity of resource j grew by 1/(1 — fij). Since r\j < ei the 
entitlements grew by a larger factor, and fulfilling them will also 
satisfy the original entitlements. 

Finally, we assume that there are no dominated inequalities, where 
an inequality xirij + ■ ■ • + xnTnj < 1 is dominated if any so- 
lution {xi, . . . ,xn) to the remaining inequalities is also a solution 
to this inequality. Dominated inequalities can be efficiently found 
by standard linear programming methods. We can clearly remove 
dominated inequalities to get a system with no dominated inequal- 
ities. Depending on the order of removal, we may end up with 
different systems. However, a solution to any of the undominated 
systems is also a solution to the original system. 

We now prove that we can find a solution a;i . . . a;jv satisfying the 
requirements of Theorem [T] under these simplifying assumptions. 
We stress that this is without loss of generality; as shown above, if 
we can find a solution under the simplifying assumptions, we can 
also find one without these assumptions. 

5.2 Proof Structure 

We first establish some notation. By ([T), the set of legal allocations 
is a subset V of R^, where 

V = { {xi, . . . ,xn) : < Xi < 1, yi and 
xiTij H \'XNrNj < 1, Vj }. 

For N = 2, this is a polygon in the first quadrant, as illustrated in 
Fig. |3] In the figure, two users contend for three resources (m = 
3). The request vectors are vi — {j, |, 1) and r2 = (1, |,0). 
This leads to the bounds shown; for example, the point (|, ii) is 
impossible because it would imply using |'3 + Y|'l = -j§of 
resource 1, i.e. more than its capacity. In the general case, this 
region is a simplex in the positive orthant (that is, the convex hull 
of A'^ affinely independent points, all in (K^)'^). 

For every vector x — {xi, . . . ,xn) in V, the set of bottleneck 
resources is 

J(x) = {j : I < j <m, xirij + ■ ■ ■ + XnTnj = 1}- 

J{x) is empty for all x in the interior of the domain V, implying 
that our solution will lie on the boundary of X>. Using this notation 
to re-write requirement our goal is to prove that there exists an 
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oH ^ 

1 Xi 

Figure 3: Depiction of bounds on Xi values due to capacity con- 
straints of resources, for N — 2 and m = 3. 

allocation x = {xi, . . . , xn), such that 

for every i ^ I,. . .,N 

there exists a j £ J{x) such that xtrij > ei. 

The difficulty in finding x stems exactly from this condition. In 
fact, if we knew what the bottleneck resources would be, the prob- 
lem could be solved efficiently using well-known machinery. Specif- 
ically, fix an arbitrary subset / C {1, . . . , m}, and consider the fol- 
lowing decision problem: Is there an x £ D for which J{x) — I 
such that condition l[3) holds? It can easily be verified that this is 
asking whether a finite set of linear equations and linear inequalities 
is consistent. This task is subsumed by the Linear Programming 
problem, and can thus be solved in polynomial time. 

How can we overcome the difficulty involved in satisfying condi- 
tion ^ without knowing in advance what the set J{x) is? We take 
a somewhat unconventional approach to this problem. The set D 
is a polytope, that is, a bounded convex subset of that is de- 
fined by a finite list of linear inequalities. We want to approximate 
!> by a subset Q C 7? that is convex and has a smooth boundary. 
Intuitively, Q "rounds off" the comers of D (see below for further 
discussion). Such a set Q is defined by infinitely many linear in- 
equalities: For every hyperplane H that is tangent to Q we write a 
linear inequality that states that x must reside "below" H. It would 
seem that this only complicates matters, replacing the finitely de- 
fined V by Q. However, the problematic condition ([3j takes on a 
much nicer form when applied to Q, and becomes a very simple 
relation involving the contact point of H and Q, the normal to H, 
and the vector e (see Equation l|7j below). Moreover, using stan- 
dard tools from the theory of ordinary differential equations, we 
can find a point on the boundary of Q where this relation holds. 

To find the solution, we do not consider a single smooth Q, but 
rather a whole parametric family Qt . This family has the properties 
that (a) the sets Qt grow as the parameter t increases; (b) they are 
all contained in I>; and (c) as t — >■ co the sets Qt converge to T). 
For every t > 0, we find a point cc'*' on the boundary of Qt such 
that a;**^ satisfies the analogue of condition As t — >■ oo the 
points cc'*' tend to the boundary of V. We argue that there always 
exists a convergent subsequence of the points x^^\ and show that 
the limit point of this subsequence solves our original problem. In 
the language of the description below, Qt is defined as the set of 
those X £T> for which f{x) < t. 



The procedure above hinges on our ability to define the appropriate 
points cc'*-' that satisfy the required condition. This is based on 
considering the tangent to the surface of Qt. Note that the only 
essential difference between D and Q is that the latter is defined 
by an infinite family of defining linear inequalities, namely, one for 
each hyperplane H that is tangent to Q. Keeping this perspective in 
mind, let us apply the original problem definition to a point x £ Q. 
If X lies in the interior of Q, then none of Q's defining inequalities 
holds with equality. Thus, as before, J{x) is empty for any x in 
the interior of the domain Q. We therefore consider x that lies on 
the boundary of Q. In this case the set J{x) is a singleton, the only 
member of which is the inequality corresponding to the hyperplane 
H that is tangent to Q and touches it at the point x. The equation of 
the tangent hyperplane H can be written as "^ftXi — 1, where the 
vector {vi, . . . ,p„) is normal to H. Now condition ([3} becomes 

Vi i^iXi > Ei. (4) 

When we sum over all i this becomes i^i^i > "12 ~ 1- But 
X lies on H, so that UiXi — 1. It follows that all inequalities 
in Eq. Q hold with equality. But we also have, from the definition 
of the bottlenecks, that "^^^ij^i ~ 1- Thus, the normal is simply 
defined by the requirements vectors. Moreover, we can use this as 
a condition on the gradients of the surfaces of Qt for successive f's, 
and follow a trajectory that leads to a solution on the boundary of 
D. This is then the desired constructive proof: it both shows that a 
solution exists, and provides a mechanism for finding it. 

5.3 The Case N = 2 

In this section, we give a complete proof of Theorem[T]for the case 
N — 2 that is simpler than our general proof, and is perhaps more 
intuitive. This includes an explanation of the relationship between 
the normals to the surfaces and the requirements vectors. The ar- 
gument for arbitrary A'^ is given in the next subsection. 

In the case A'^ = 2, as noted above, the constraint Q defines a 
region in the first quadrant whose boundary is a piecewise linear 
curve that satisfies the constraints in ([TJ with < replaced by =. 
Note that the slopes of the lines that define the boundary are neg- 
ative, and as k increases from to 1, the slopes of the lines that 
intersect the vertical line x — k get more and more negative. This 
follows from the fact that the interior is convex. 

Let g be the piecewise linear curve that defines the boundary. We 
can approximate g arbitrarily closely from below by a concave 
twice-differentiable function /. (The function g is the boundary 
of the region called © in the previous section; the function / is the 
boundary of the region Q.) The concavity of the curve / just means 
that /" < 0. As we said earlier, / "rounds off" the comers of g, as 
shown in Fig.|4] 

As we said in the previous section, we want to find a point (a;* , ) 
on the curve / such that if viXi + U2X2 = 1 is the tangent to the 
curve at [x\, X2), then i^ix* = ei and V2X2 = 62. We show below 
how to find such a point. We now argue that finding such a point 
for each / approximating g suffices to prove the theorem in the 
case that N = 2. First suppose that (2;*, X2) is actually a point on 
one of the lines that define g (as opposed to a point on / that arises 
from rounding off a corner of the curve g). Suppose that the line is 
defined by resource j, so that it has the form rijXi + r2jX2 ~ 1. 
Obviously the tangent to / at the point {xl,X2) on the line is just 
the line itself, so we have i/i ~ rij and P2 ~ r2j. Thus, we will 
have found xl and X2 such that rijxl — ei and r2jX2 = 62, which 
means that ^ holds (and moreover, the same resource provides 
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Figure 4: Rounding off the boundary ofT>. 



justification for both users). 

Next, suppose that (3:1,2:2) is not on one of the original lines, 
but we can find such a point for all functions / approximating g. 
Straightforward continuity arguments show that small changes to / 
result in small changes to the point, so that as / gets closer to g, we 
get a sequence of points that approach a point on g. Thus the limit 
is a point on g. We actually get even more. What we really have 
for each function / that approximates g is two pairs {x\,X2) and 
(i^i , 2^2), where (a;i , 2:2) is apoint on /, uix\-\-V2X2 ~ lis the tan- 
gent to / at (3:1, 2:2), fixl — ei, and 1^22:2 ~ £2. As / approaches 
g, we will get a sequence of such pairs of points. Let {xf, X2) and 
{vf, 1^2) be the limit of this sequence of pairs of pairs. It is clearly 
the case that {xf,X2) is a point on g, u^xf = ei, ^2^2 ~ ^2- ™d 
ViXi + 1^2X2 = 1. Now if (2;j , 2:2) is in the interior of one of the 
lines that make up the boundary of the region — let's assume it is 
the line associated with resource j — then, as the argument above 
suggests, = rij and = r2j. Thus, resource j is a bottleneck, 
and provides a justification for both users. 

Now suppose that (2:5', 2:2) is at the intersection of two lines, say, 
representing resources j and j' . Thus, xfrij + 2;|r-2j = 1 and 
+2;2'"2j' = 1> so both resources are bottlenecks at (x^,2;|). 
Moreover, we still have i/fxf = ei and i^2^2 ~ ^2. Finally, it 
is clear that must be a convex combination of Vij and r^j/, for 
i G {1, 2}, since, for each approximation / to g, the tangent in the 
region that we have "rounded off" is a convex combination of the 
tangents of the lines that make up g that are being approximated. 
It follows that each user i G {1,2} gets at least his entitlement on 
one of resources j or j'; that is, either rijxf > Ci or Tijix^ > ei. 
For if Tijxl < Ei and rijixf < e;, then v^xf < ei, and we have a 
contradiction. 

The fact that we can find such a point (2:1, 2:2) on each / fol- 
lows from another easy continuity argument. Consider the points 
on the function / in the first quadrant. Suppose that / starts at 
the F-axis at some point (0, y') and ends at the X-axis at some 
point (2;',0). Let the equation of the tangent of / at the point 
a;* = {xi,X2) he u'^ ■ x — 1. Consider the term q{x°) = 
{vfx\)/{vfxl) = ~f'(xt)xt/xt as x'> goes from {0,y') to 
{x' , 0). As a;* approaches (0, y') from the right, (?(a;*) approaches 
0; as a;* approaches (2;',0) from the left, q{x°) approaches 00. 
Since /' is continuous, q varies continuously in the first quadrant 
between and oo. Thus, at some point it must have value ei/e2. If 
q{x*) = ei/e2, then we must have i^f x\/v2 ^2 ~ ei/e2. Since 
we also have vf x* + 1^2 X2 ~ 1 and ei + 62 = 1, it easily fol- 
lows that we must have i/f xl = ei and i^f 2^2 = 62, as desired. 



point corresponding 




0.6 1 Xi 



Figure 5: Simple example of a point (2:1, 2:2). Different points 
correspond to different ratios ^ , as indicated by the slopes of the 
line segments. 

This completes the proof in the case that N = 2. 

We can actually say more in the case that N = 2. Since / is con- 
cave, /' is decreasing, so — /' is increasing. It easily follows that 
q is an increasing function. Thus, there is a unique point (x*, 2:2) 
with the desired properties. It easily follows that, in the case of 
two users, the solution to ([TJ and Q is unique. Uniqueness has 
an important consequence. In the problem definition, the bottle- 
neck resources in QJ are not known in advance. In particular, it 
might seem that different solutions may lead to different resources 
becoming bottlenecks. Uniqueness guarantees that this is not the 
case, and that the set of resources that will become bottlenecks is 
uniquely defined by the problem parameters (that is, the entitle- 
ments and request profiles). We remark that the uniqueness claim 
does not hold in general for N > 2; see Section [53] 

The example in Fig. |5] may help in gaining an intuition for the 
derivation above. Consider a single limiting resource, where both 
users request ri = r2 = | of its capacity. The boundary line 
representing the capacity limit of the resource has the equation 
|xi + |22 = 1. Different points along this line correspond to 
different ratios of the users' entitlements. For example, if they have 
entitlements of 0.4 and 0.6, the point (0.6, 0.9) satisfies the equa- 
tions I ■ 0.6 = 0.4 and | ■ 0.9 = 0.6, and indeed using these 
values for 2:1 and 2:2 leads to sharing the resource in the desired 
proportions. If the ratio of entitlements is such that ^ > 2, then 
user 2 is not requesting his full entitlement, and is eliminated from 
consideration. He is given his full request, and user 1 gets the rest, 
which is more than his entitlement (so they are both satisfied). This 
is in fact an example of a solution based on a dummy resource. The 
opposite happens if < ^ • 

5.4 Proof of Theorem [T] 

We now prove Theorem[T]for arbitrary A'^. 

Construction 1. To every allocation x in the interior of the domain 
D, we assign a value 

m / N \ 

/(a;) = -^log l-^2-,rfc, . (5) 

j=i V fc=i / 

Remark 1. The function / is positive in the interior of V, diverging 
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Figure 6: Illustration of level-sets of f from t — tot = oo. 

to infinity as x tends to the boundary of ©13 

Remark 2. Clearly, there are other choices of / that satisfy these 
desired properties. This choice seems like the simplest one for our 
purposes. 

Definition 1. To every number t > 0, there corresponds a level set 
of /, namely, 

Remarks. This is an (TV — 1) -dimensional hypersurface. (Fig. [6] 
illustrates this for iV = 2.) 

Definition 2. To every point x £ T>, there corresponds a unique 
unit vector vix) — {i'i{x), . . . , vn{x)), normal to the level set of 
/at a;. 

The unit normal ^{x) is proportional to the gradient of / at x, 
implying that 



\fi = 1, 



(6) 



where the normalization constant c is chosen so as to guarantee that 



f is a unit vector, that is, i^i + ■ 



1. 



Construction 2. We now construct a vector- valued function 

X{t) = {xi{t),...,XN{t)), t>0, 

satisfying the following properties; 

1. x{t) lies on the level set Ft for alH > (and, in particular, 
remains in ©). 

2. For all t > 0, there exists a t-dependent normalization factor 
c{t), such that for every i = 1, . . . , N, 



Xi{t) Ui{x{t)) = cc{t)ei. 



(V) 



Remark 4. Note that since /(a;(0)) = it follows that x{0) = 0, 
that is, the vector-valued function x{t) "starts" at the origin. 

Remark 5. substituting ^ into ([7J and summing over the index i 
determines cc{t). After simple algebraic manipulations, summing 
the expressions Q over i gives us 



y> Xi{t)rij - (Efc^ia:fc(^)r-fcj)ei 
3=1 -J2k=i^k{t)rkj 



0, Vi 



,N, \/t> 0. 
(8) 



Intuitively, x{t) is a "trajectory" that takes us from the origin x — 
to a point on the boundary of O as t grows from to oo. 

The formal proof now follows from the following sequence of three 
lemmas, proved below. First, we show that a trajectory with the 
required properties exists (Lemma|4ll. Given such a trajectory, we 
show that a subsequence of this trajectory converges to a point on 
the boundary of T) (Lemma|2](. Finally, this accumulation point is 
shown to be a solution to our allocation problem (Lemma[3}. 

It is convenient to delay the question of whether there indeed exists 
a trajectory x{t) satisfying the required properties, and consider 
convergence first. 

Lemma 2. Let < ti < t2 < ■ ■ ■ be a sequence tending to infin- 
ity. Let x{t) be a vector-valued function as defined in Construction 
|2] Then, the sequence x(ti) has a subsequence that converges to 
an allocation x* on the boundary ofT>. 

Proof. Consider what happens as t — >■ oo. Since x{t) € 
Ft, it follows that x{t) approaches the boundary of "D. However, 
the function x{t) may not tend to a limit as t — > oo. Neverthe- 
less, since D is a compact domain, x(t) has a convergent subse- 
quence. That is, there exists an allocation x* = {xX, . . . , x*pf) on 
the boundary of 7? and a subsequence t„i < tn^ < ■ ■ ■ such that 



lim x{tn^) — X* . 



□ 



The next lemma shows that this accumulation point is a solution to 
the fair allocation problem. 

Lemma 3. An allocation x* as resulting from Lemma^is a fair 
allocation according to our definition. 

Proof. Since X* is on the boundary of D, it has a non-empty 
set J{x*) of bottleneck resources such that 

x*irij + ■ ■ ■ + x*MrNj = 1 V j G Jix") / 0. 

We then rewrite ^ by splitting the resources j into bottleneck re- 
sources and non-bottleneck resources, and setting t — t„: 

^^7? : ^ r 



(9) 



E 

jeJ{a;*) 



= 0. 



The two summations behave very differently as n — >■ oo. For a non- 
bottleneck resource j, X^feLi ^l^'kj < 1, so the summation over 
the non-bottleneck resources tends to a limit obtained by letting 
x{tn) X* term-by-term: 



lim y 



Xi{t„)rij - iT,k=i^>'{tn)rkj)ei 



1 - J2k = l^kitn)rkj 
y ~ (X^fc=l ^k'''kj)e.i 



(10) 



1 Ylik=l ^k'''kj 



"Note that this function / is not the curve / of Section [53 



For a bottleneck resource j, the denominator 1 — X^^i 3:krkj tends 
to zero as X a-* , so the limit exists only if the numerator vanishes 
as well. But if it were the case that, for a given user i, 

x*rij < Bi for all j G J{x*), 
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then 



lim 



E 



1 - T,k = l^k{tn)rkj 



jeJ(iE*) 

This is a contradiction to the fact that, by the limit should be 
the negative of the right-hand side of dlOt . Hence we conclude that 
X* has the property that for all users i, there exists a bottleneck 
resource j such that x*rij > ei. Thus, x* is a fair allocation. □ 

It remains to show that the trajectory x {t) is indeed well-defined for 
all system parameters ei and r^j . This is handled by the following 
lemma. 

Lemma 4. There exists a function x(t) with the properties speci- 
fied in Construction^ 

Proof. To prove this we show that we can find points satisfying 
property 1 that also satisfy property 2. Since x{t) G Ft, we have 
f{x{t)) = t, that is. 



E^°S 1 ~^Xk{t)rkj 



By Q, 

m 



Xi{t)ri: 



^ 1 -Efe=l2;fe(^)^■fcJ 



= c{t)ei, \/i = 1, 



(11) 



,N. (12) 



Differentiating both equations with respect to t, we obtain a linear 
system of equations for the derivative dx/dt. Differentiating jl lb . 
we get 



Differentiating il2\ . we get 



V^JV """Z^ v^iV NO ^ rli ' 

J2k=i^krkj ,=1 (1-Efe=ia;fcrfcj)2 c(t 



Observe that, without loss of generality, we can set dc/dt — 1, 
compute the resulting vector of derivatives dx/dt, and then multi- 
ply it by a constant for the normalization condition to hold. Thus, it 
remains only to show that ( 113b has a unique solution when dc/dt — 
1. To do so, we define an cc-dependent matrix with entries 



i = 1, 



i = 1, • 



These entries are non-negative for x £T>. We now rewrite l ll3b in 
a more compact form. 



dt 



kj 



i=i 



The term inside the brackets is the {k, i) entry of a symmetric 
positive-definite N x N matrix, which immediately implies that 
there exists a unique solution dx/dt. Moreover, since the depen- 
dence of dx/dt on X is continuous, the existence and uniqueness 
of x{t) follows from the Fundamental Theorem of Ordinary Differ- 
ential Equations (9]. (More precisely, the fundamental theorem of 
ODEs guarantees only the existence and uniqueness of a solution 
for some small t; global existence follows from the boundedness of 
the domain C) □ 

This completes the proof of Theorem[T] 



We note that our proof that a fair allocation exists is almost con- 
structive. The trajectories x{t) can easily be computed numerically 
using standard ODE integrators (for example, Matlab's ode 4 5 func- 
tion). If x{t) is found to tend to a limit for large t, then this limit is 
a fair allocation. The only reservation is that numerical integration 
only provides approximate solutions (however, with a controllable 
error), and can only be carried out over a finite t interval. 

5.5 Uniqueness of the Solution 

As we mentioned in Section [531 unlike the case A'' = 2, in the 
general case the solution is not unique. This is easily seen from 
the following counterexample. Assume A'^ = 3 and m = 2, with 
r-i = (1,1), ra = (0, 1), rs = (1,0), and e = (0.5,0.3,0.2). 
This has the the family of solutions x — {z,l — z,l — z) for z 
that satisfies 0.5 < z < 0.7, where in all these solutions both 
resources are bottlenecks. Note that this does not contradict the fact 
that our solution method finds a unique trajectory. This trajectory 
corresponds to the choice of the function / in l[5]l. Other choices, 
e.g. by adding different weighting factors to each term in the sum, 
could lead to other trajectories and other solutions. 

There also exist cases where different solutions depend on different 
bottlenecks. Consider the following example, with four users and 
four resources (A'^ — m = 4). Assume all users have the same enti- 
tlements, that is Ei = 0.25 for i = 1, . . . , 4. Arrange the users and 
resources in a circle, and make each user request the full capacity 
of its resource and those of its neighbors. Thus the requirements 
matrix becomes 

/ 1 1 1 \ 
1110 
111 
10 11 



This instance is completely symmetric, and the obvious solution is 
a symmetric allocation where = | for i = 1, . . . , 4. In this so- 
lution, all 4 resources are bottlenecks, and all users get more than 
their entitlements on all the resources they use. But there are 6 ad- 
ditional solutions. Pick any two users i and j, and set Xi — Xj — 
0.25. Let k and I be the other two users, and set Xk = xi = 0.375. 
Now two resources are bottlenecks (0.25 + 0.375 + 0.375 = 1) 
but the other two are not (0.25 + 0.25 + 0.375 = 0.875). Which 
resources become bottlenecks depends on the choice of k and I. 
If they are adjacent, then resources k and I are the bottlenecks. If 
they are opposite each other, then resources i and j are the bottle- 
necks. In any case, every user gets his entitlement on at least one 
bottleneck resource. This demonstrates that the set of bottleneck 
resources is not unique. 

The finding that there may be multiple solutions opens the issue of 
selecting among them. In particular, once one accepts our defini- 
tion of fairness and finds a set of fair solutions that satisfy all users, 
it becomes possible to use the remaining freedom to select the spe- 
cific solution that optimizes some other metric. For example, we 
can decide that the secondary goal is to maximize system utiliza- 
tion; in the above example, this will lead to preferring the sym- 
metric solution where all resources are bottlenecks over the other 
solutions where only two are bottlenecks. This provides an inter- 
esting way to combine user-centric metrics (the entitlements) with 
system-centric metrics (resource utilization). 

Of course, making such optimizations hinges on our ability to iden- 
tify and characterize all the possible solutions. At present how to 
do this remains an open question. 
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6. CONCLUSIONS 

To summarize, our main contribution is the definition of what it 
means to make a fair allocation of multiple continuously-divisible 
resources when users have different requirements for the resources, 
and a proof that such an allocation is in fact achievable. The defini- 
tion is based on the identification of bottleneck resources, and the 
allocation guarantees that each user either receives all he wishes 
for, or else gets at least his entitlement on some bottleneck resource. 
The proof is constructive in the sense that it describes a method to 
find such a solution numerically. The method has in fact been pro- 
grammed in Matlab, and was used in our exploration of various 
scenarios. While this method has seemed efficient in practice, one 
obvious open question is whether we can get a method that is poly- 
nomial in N and m. 

Note that, in the context of on-line scheduling, we may not need 
to find an explicit solution in advance. Consider for example the 
RSVT scheduler described by Ben-Nun et al. (5|. This is a fair 
share scheduler that bases scheduling decisions on the gap between 
what each user has consumed and what he was entitled to receive. 
To do so, the system keeps a global view of resource usage by the 
different users. If there is only one bottleneck in the system, this 
would be applied to the bottleneck resource. The question is what 
to do if there are multiple bottlenecks. Our results indicate that 
the correct course of action is to prioritize each process based on 
the minimal gap on any of the bottleneck devices, because this is 
where it is easiest to close the gap and achieve the desired entitle- 
ment. Once the user achieves his target allocation on any of the 
bottleneck devices, he should not be promoted further. This con- 
tradicts the intuition that when a user uses multiple resources, his 
global priority should be determined by the one where he is farthest 
behind. 

It should also be noted that our proposal pertains to the policy level, 
and only suggests the considerations that should be applied when 
fair allocations are desired. It can in principle be used with any 
available mechanism for actually controlling resource allocation, 
for example, resource containers (3). 

A possible direction for additional work is to extend the model. In 
particular, an interesting question is what to do when the relative 
usage of different resources is not linearly related. In such a case, 
we need to replace the user-based factors Xi by specific factors Xij 
for each user and resource. This also opens the door for a game 
where users adjust their usage profile in response to system allo- 
cations — for example, substituting computation for bandwidth by 
using compression — and the use of machine learning to predict 
performance and make optimizations (Q. Finally, we might con- 
sider approaches where users have specific utilities associated with 
each resource. 
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